Evaluation des systèmes mobiles et ubiquitaires: proposition de méthodologie et retours d'expérience

International audienceThe evaluation of mobile and/or ubiquitous interactive systems via user testing seems a priori more relevant in the field than in a usability laboratory. However, the results of the literature are contradictory. In this article, we aim at explaining the reasons why, and we propose a methodology that could minimize biases. The experiments described in the literature and our own experiments lead us to define the interactive environment concept and three possible experimental approaches: laboratory, field and reality testing. Then, we propose a methodology and a technique -the Trojan horse- adapted to the evaluation in reality testing. At last, we illustrate the theoretical approach by three experiments and give experience feedbacks on them. We conclude on the limits of our approach.L'évaluation des systèmes interactifs mobiles et/ou ubiquitaires par l'intermédiaire des tests utilisateurs semble a priori plus pertinente sur le terrain qu'en laboratoire d'utilisabilité. Pourtant, les résultats de la littérature apparaissent comme contradictoires. Notre objectif dans cet article est d'en expliciter les raisons et de proposer une méthodologie minimisant les biais. Les expérimentations décrites dans la littérature et nos propres travaux nous ont amenés à définir le concept d'environnement interactif et trois approches expérimentales possibles : laboratoire, terrain et situation réelle. Nous proposons ensuite une méthodologie et une technique (le cheval de Troie) adaptées à l'évaluation en situation réelle. Enfin, nous illustrons notre approche théorique par trois expérimentations et en donnons des retours d'expérience. Nous concluons ensuite sur les limites de notre approche

Design and evaluation of a smart home voice interface for the elderly ― Acceptability and objection aspects

Impact-F=1.13 estim. in 2012International audienceSmart homes equipped with ambient intelligence technology constitute a promising direction to enable the growing number of elderly to continue to live in their own home as long as possible. However, this calls for technological solutions that suit their specific needs and capabilities. The SWEET-HOME project aims at developing a new user friendly technology for home automation based on voice command. This paper reports a user evaluation assessing the acceptance and fear of this new technology. Eight healthy persons between 71 and 88 years old, 7 relatives (child, grandchild or friend) and 3 professional carers participated in a user evaluation. During about 45 min, the persons were questioned in co-discovery in the DOMUS smart home alternating between interview and wizard of Oz periods followed by a debriefing. The experience aimed at testing four important aspects of the project: voice command, communication with the outside world, domotics system interrupting a person's activity, and electronic agenda. Voice interface appeared to have a great potential to ease daily living for elderly and frail persons and would be better accepted than more intrusive solutions. By considering still healthy and independent elderly people in the user evaluation, an interesting finding that came up is their overall acceptance provided the system does not drive them to a lazy lifestyle by taking control of everything. This particular fear must be addressed for the development of smart homes that support daily living by giving them more ability to control rather than putting them away from the daily routine

The Sweet-Home speech and multimodal corpus for home automation interaction

International audienceAmbient Assisted Living aims at enhancing the quality of life of older and disabled people at home thanks to Smart Homes and Home Automation. However, many studies do not include tests in real settings, because data collection in this domain is very expensive and challenging and because of the few available data sets. The SWEET-H OME multimodal corpus is a dataset recorded in realistic conditions in D OMUS, a fully equipped Smart Home with microphones and home automation sensors, in which participants performed Activities of Daily living (ADL). This corpus is made of a multimodal subset, a French home automation speech subset recorded in Distant Speech conditions, and two interaction subsets, the first one being recorded by 16 persons without disabilities and the second one by 6 seniors and 5 visually impaired people. This corpus was used in studies related to ADL recognition, context aware interaction and distant speech recognition applied to home automation controled through voice

DynEmo: A video database of natural facial expressions of emotions.

International audienceDynEmo is a database available to the scientific community (https://DynEmo.liglab.fr/). It contains dynamic and natural emotional facial expressions (EFEs) displaying subjective affective states rated by both the expresser and observers. Methodological and contextual information is provided for each expression. This multimodal corpus meets psychological, ethical, and technical criteria. It is quite large, containing two sets of 233 and 125 recordings of EFE of ordinary Caucasian people (ages 25 to 65, 182 females and 176 males) filmed in natural but standardized conditions. In the Set 1, EFE recordings are associated with the affective state of the expresser (self-reported after the emotion inducing task, using dimensional, action readiness, and emotional labels items). In the Set 2, EFE recordings are both associated with the affective state of the expresser and with the time line (continuous annotations) of observers' ratings of the emotions displayed throughout the recording. The time line allows any researcher interested in analysing non-verbal human behavior to segment the expressions into emotions

Evaluation of a context-aware voice interface for Ambient Assisted Living: qualitative user study vs. quantitative system evaluation

International audienceThis paper presents an experiment with seniors and people with visual impairment in a voice-controlled smart home using the SWEET-HOME system. The experiment shows some weaknesses in automatic speech recognition which must be addressed, as well as the need of better adaptation to the user and the environment. Indeed, users were disturbed by the rigid structure of the grammar and were eager to adapt it to their own preferences. Surprisingly, while no humanoid aspect was introduced in the system, the senior participants were inclined to embody the system. Despite these aspects to improve, the system has been favourably assessed as diminishing most participant fears related to the loss of autonomy

The EEE corpus: socio-affective "glue" cues in elderly-robot interactions in a Smart Home with the EmOz platform

International audienceThe aim of this preliminary study of feasibility is to give a glance at interactions in a Smart Home prototype between the elderly and a companion robot that is having some socio-affective language primitives as the only vector of communication. The paper particularly focuses on the methodology and the scenario made to collect a spontaneous corpus of human-robot interactions. Through a Wizard of Oz platform (EmOz), which was specifically developed for this issue, a robot is introduced as an intermediary between the technological environment and some elderly who have to give vocal commands to the robot to control the Smart Home. The robot vocal productions increases progressively by adding prosodic levels: (1) no speech, (2) pure prosodic mouth noises supposed to be the "glue's" tools, (3) lexicons with supposed "glue" prosody and (4) subject's commands imitations with supposed "glue" prosody. The elderly subjects' speech behaviours confirm the hypothesis that the socio-affective "glue" effect increase towards the prosodic levels, especially for socio-isolated people. The actual corpus is still on recording process and is motivated to collect data from socio-isolated elderly in real need

The EEE corpus: socio-affective "glue" cues in elderly-robot interactions in a Smart Home with the EmOz platform

International audienceThe aim of this preliminary study of feasibility is to give a glance at interactions in a Smart Home prototype between the elderly and a companion robot that is having some socio-affective language primitives as the only vector of communication. The paper particularly focuses on the methodology and the scenario made to collect a spontaneous corpus of human-robot interactions. Through a Wizard of Oz platform (EmOz), which was specifically developed for this issue, a robot is introduced as an intermediary between the technological environment and some elderly who have to give vocal commands to the robot to control the Smart Home. The robot vocal productions increases progressively by adding prosodic levels: (1) no speech, (2) pure prosodic mouth noises supposed to be the "glue's" tools, (3) lexicons with supposed "glue" prosody and (4) subject's commands imitations with supposed "glue" prosody. The elderly subjects' speech behaviours confirm the hypothesis that the socio-affective "glue" effect increase towards the prosodic levels, especially for socio-isolated people. The actual corpus is still on recording process and is motivated to collect data from socio-isolated elderly in real need

The SWEET-HOME Project: Audio Technology in Smart Homes to improve Well-being and Reliance

International audienceThe SWEET-HOME project aims at providing audio-based interaction technology that lets the user have full control over their home environment, at detecting distress situations and at easing the social inclusion of the elderly and frail population. This paper presents an overview of the project focusing on the implemented techniques for speech and sound recognition as context-aware decision making with uncertainty. A user experiment in a smart home demonstrates the interest of this audio-based technology

Utilisation d’outils oculométriques dans le repérage d’indices visuels de la LSF en perception

International audienceUsing eye-tracking tools is a new platform in exploring sign language linguistics. In this paper, we present two pilot experiments which were conducted at different levels of language analysis, phonetic and morpho-syntactic, and assess the potential of these tools from measurements of fixations and eye movements for signers.L ’ utilisation d’ outils oculométriques constitue un nouveau tremplin dans l’ exploration linguistique des langues des signes. Nous présentons ici deux expérimentations pilotes qui ont été menées à des niveaux distincts de l’analyse linguistique, phonétique et morpho-syntaxique, et évaluons les possibilités offertes par ces outils à partir de mesures de fixations et de trajets oculaires chez des signeurs et des non- signeurs